Fixing missed arguments for pools when init transports by NewUserHa · Pull Request #2997 · encode/httpx

NewUserHa · 2023-12-09T20:18:54Z

Summary

Lines 258 to 330 in 5b5f6d8

    
           class AsyncHTTPTransport(AsyncBaseTransport): 
        
               def __init__( 
        
                   self, 
        
                   verify: VerifyTypes = True, 
        
                   cert: typing.Optional[CertTypes] = None, 
        
                   http1: bool = True, 
        
                   http2: bool = False, 
        
                   limits: Limits = DEFAULT_LIMITS, 
        
                   trust_env: bool = True, 
        
                   proxy: typing.Optional[Proxy] = None, 
        
                   uds: typing.Optional[str] = None, 
        
                   local_address: typing.Optional[str] = None, 
        
                   retries: int = 0, 
        
                   socket_options: typing.Optional[typing.Iterable[SOCKET_OPTION]] = None, 
        
               ) -> None: 
        
                   ssl_context = create_ssl_context(verify=verify, cert=cert, trust_env=trust_env) 
        
                   if proxy is None: 
        
                       self._pool = httpcore.AsyncConnectionPool( 
        
                           ssl_context=ssl_context, 
        
                           max_connections=limits.max_connections, 
        
                           max_keepalive_connections=limits.max_keepalive_connections, 
        
                           keepalive_expiry=limits.keepalive_expiry, 
        
                           http1=http1, 
        
                           http2=http2, 
        
                           uds=uds, 
        
                           local_address=local_address, 
        
                           retries=retries, 
        
                           socket_options=socket_options, 
        
                       ) 
        
                   elif proxy.url.scheme in ("http", "https"): 
        
                       self._pool = httpcore.AsyncHTTPProxy( 
        
                           proxy_url=httpcore.URL( 
        
                               scheme=proxy.url.raw_scheme, 
        
                               host=proxy.url.raw_host, 
        
                               port=proxy.url.port, 
        
                               target=proxy.url.raw_path, 
        
                           ), 
        
                           proxy_auth=proxy.raw_auth, 
        
                           proxy_headers=proxy.headers.raw, 
        
                           ssl_context=ssl_context, 
        
                           max_connections=limits.max_connections, 
        
                           max_keepalive_connections=limits.max_keepalive_connections, 
        
                           keepalive_expiry=limits.keepalive_expiry, 
        
                           http1=http1, 
        
                           http2=http2, 
        
                           socket_options=socket_options, 
        
                       ) 
        
                   elif proxy.url.scheme == "socks5": 
        
                       try: 
        
                           import socksio  # noqa 
        
                       except ImportError:  # pragma: no cover 
        
                           raise ImportError( 
        
                               "Using SOCKS proxy, but the 'socksio' package is not installed. " 
        
                               "Make sure to install httpx using `pip install httpx[socks]`." 
        
                           ) from None 
        
                       self._pool = httpcore.AsyncSOCKSProxy( 
        
                           proxy_url=httpcore.URL( 
        
                               scheme=proxy.url.raw_scheme, 
        
                               host=proxy.url.raw_host, 
        
                               port=proxy.url.port, 
        
                               target=proxy.url.raw_path, 
        
                           ), 
        
                           proxy_auth=proxy.raw_auth, 
        
                           ssl_context=ssl_context, 
        
                           max_connections=limits.max_connections, 
        
                           max_keepalive_connections=limits.max_keepalive_connections, 
        
                           keepalive_expiry=limits.keepalive_expiry, 
        
                           http1=http1, 
        
                           http2=http2, 
        
                       ) 
        
                   else:  # pragma: no cover

eg.

in AsyncHTTPTransport, AsyncConnectionPool has the argument of retries passed in, but AsyncHTTPProxy and AsyncSOCKSProxy have not;
in HTTPTransport, it has the same issue.

discussion: #2988

This also requires a fix in https://github.com/encode/httpcore

Checklist

I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
I've updated the documentation accordingly.

NewUserHa · 2023-12-10T00:59:36Z

The PR requires encode/httpcore#857 PR to pass checks

lovelydinosaur · 2023-12-11T13:54:43Z

The PR requires encode/httpcore#857 PR to pass checks

Can we just start by considering retries here? That's already supported by httpcore>1.0, right?

NewUserHa · 2023-12-11T19:54:36Z

Ok, done.

Should open a new PR for the rest missed arguments?

lovelydinosaur · 2023-12-12T13:46:41Z

Okay, first up let's just have a sanity check... do connection retries actually get applied when connecting to proxies?

NewUserHa · 2023-12-12T14:33:15Z

The SocksProxy?

Fixing missed arguments for pools when init transports

180af89

NewUserHa mentioned this pull request Dec 9, 2023

Fixing missed arguments for pools when init transports encode/httpcore#857

Closed

3 tasks

NewUserHa added 2 commits December 10, 2023 08:51

update

f90ca24

fix lint

3fa6052

NewUserHa added 3 commits December 12, 2023 03:50

leave uds and local_address of socksproxy for antoher PR

45e5f14

leave socket_options for another PR

30a8c7b

reorder retries, `local_address, uds

cb50ff8

Merge branch 'master' into patch-1

96ffa64

NewUserHa added 3 commits December 14, 2023 05:02

Merge branch 'master' into patch-1

8d55b09

fix

78ac27a

Merge branch 'master' into patch-1

2b11bcd

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fixing missed arguments for pools when init transports#2997

Fixing missed arguments for pools when init transports#2997
NewUserHa wants to merge 10 commits intoencode:masterfrom
NewUserHa:patch-1

NewUserHa commented Dec 9, 2023

Uh oh!

NewUserHa commented Dec 10, 2023

Uh oh!

lovelydinosaur commented Dec 11, 2023

Uh oh!

NewUserHa commented Dec 11, 2023

Uh oh!

lovelydinosaur commented Dec 12, 2023

Uh oh!

NewUserHa commented Dec 12, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	class AsyncHTTPTransport(AsyncBaseTransport):
	def __init__(
	self,
	verify: VerifyTypes = True,
	cert: typing.Optional[CertTypes] = None,
	http1: bool = True,
	http2: bool = False,
	limits: Limits = DEFAULT_LIMITS,
	trust_env: bool = True,
	proxy: typing.Optional[Proxy] = None,
	uds: typing.Optional[str] = None,
	local_address: typing.Optional[str] = None,
	retries: int = 0,
	socket_options: typing.Optional[typing.Iterable[SOCKET_OPTION]] = None,
	) -> None:
	ssl_context = create_ssl_context(verify=verify, cert=cert, trust_env=trust_env)

	if proxy is None:
	self._pool = httpcore.AsyncConnectionPool(
	ssl_context=ssl_context,
	max_connections=limits.max_connections,
	max_keepalive_connections=limits.max_keepalive_connections,
	keepalive_expiry=limits.keepalive_expiry,
	http1=http1,
	http2=http2,
	uds=uds,
	local_address=local_address,
	retries=retries,
	socket_options=socket_options,
	)
	elif proxy.url.scheme in ("http", "https"):
	self._pool = httpcore.AsyncHTTPProxy(
	proxy_url=httpcore.URL(
	scheme=proxy.url.raw_scheme,
	host=proxy.url.raw_host,
	port=proxy.url.port,
	target=proxy.url.raw_path,
	),
	proxy_auth=proxy.raw_auth,
	proxy_headers=proxy.headers.raw,
	ssl_context=ssl_context,
	max_connections=limits.max_connections,
	max_keepalive_connections=limits.max_keepalive_connections,
	keepalive_expiry=limits.keepalive_expiry,
	http1=http1,
	http2=http2,
	socket_options=socket_options,
	)
	elif proxy.url.scheme == "socks5":
	try:
	import socksio # noqa
	except ImportError: # pragma: no cover
	raise ImportError(
	"Using SOCKS proxy, but the 'socksio' package is not installed. "
	"Make sure to install httpx using `pip install httpx[socks]`."
	) from None

	self._pool = httpcore.AsyncSOCKSProxy(
	proxy_url=httpcore.URL(
	scheme=proxy.url.raw_scheme,
	host=proxy.url.raw_host,
	port=proxy.url.port,
	target=proxy.url.raw_path,
	),
	proxy_auth=proxy.raw_auth,
	ssl_context=ssl_context,
	max_connections=limits.max_connections,
	max_keepalive_connections=limits.max_keepalive_connections,
	keepalive_expiry=limits.keepalive_expiry,
	http1=http1,
	http2=http2,
	)
	else: # pragma: no cover

Uh oh!

Conversation

NewUserHa commented Dec 9, 2023

Summary

Checklist

Uh oh!

NewUserHa commented Dec 10, 2023

Uh oh!

lovelydinosaur commented Dec 11, 2023

Uh oh!

NewUserHa commented Dec 11, 2023

Uh oh!

lovelydinosaur commented Dec 12, 2023

Uh oh!

NewUserHa commented Dec 12, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants